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Abstract 

We prove a sufficient stochastic maximum principle for the optimal control of a regime- 
switching diffusion model. We show the connection to dynamic programming and we apply the 
result to a quadratic loss minimization problem, which can be used to solve a mean-variance 
{^ . portfolio selection problem. 
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en ■ 1 Introduction 

>. 

The aim of this paper is to prove a sufficient stochastic maximum principle for optimal control 
within a regime-switching diffusion model. This extends the result of lFramstad et alJ 120041 . which 
is in a jump-diffusion setting. To prove this, we follow the method in lFramstad et al.l [2004] . As in 
their paper, we show the connection to dynamic programming and show how to apply the result to 
£— v ■ a quadratic loss minimization problem. 

An early maximum principle for a diffusion model is in lBismutl 1973J . where a necessary maxi- 



(N 



>< 



mum principle is derived in a model which is somewhat structurally similar to our own and, as we 
also find in our set-up, this results in jumps in the adjoint va riables of the Hamiltonian. 

For a hidden Markovian regime-switching diffusion model, Elliott et al.l 2010J apply, though do 



not state explicitly, a sufficient maximum principle to a mean-variance portfolio selection problem. 
However, their model is not the same as the one we consider and hence they do not obtain jumps 
in the adjoint variables. 

In Section [5] we detail the regime-switching diffusion model and in Section [3] we set out the 
control problem. The sufficient stochastic maximum principle is given in Section 2] This is followed 
by demonstrating in Section [5] the connection with dynamic programming. Finally in Section [6] 
we illustrate the use of the sufficient stochastic maximum principle by solving a quadratic loss 
minimization problem. 



* (CA.Donnelly9hw.ac.uk). Mailing address: Department of Actuarial Mathematics and Statistics, Heriot-Watt 
University, Edinburgh, U.K. Phone: +44 131 451 3251. Fax: +44 131 451 3249. 
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2 The regime-switching diffusion model 

Let T £ (0, oo) be a fixed, deterministic time. We assume that we are given an iV-dimensional 
Brownian motion W = {W\, ■ ■ ■ , W)v) and a continuous-time, finite state space Markov chain a 
defined on the same probability space (£2, T 1 P). 

The filtration is generated jointly by the Brownian motion W and the Markov chain a, 

F t :=a{(a(s),W(s)),s€lO,t}}VAf(V), Vie[0,T], (2.1) 

where W(P) denotes the collection of all P-null events in the probability space (£2, T, P). 

We assume that the Markov chain takes values in a finite state space I — {1, . . . ,D} and it 
starts in a fixed state io € /, so that a(0) = io, a.s. The Markov chain a has a generator G which is 
a D x D matrix G — (ffy)fj=i- Denote by 1 the zero-one indicator function. Associated with each 
pair of distinct states (i,j) in the state space of the Markov chain is a point process, or counting- 
process, 

Nij(t):= J2 1 {a(-_)=i}l{a(.)=i}, We[0,T]. (2.2) 

0<s<t 

The process Nij(t) counts the number of jumps that the Markov chain a has made from state i to 
state j up to time t. Define the intensity process 

Aij(t) :=5ijl{a(t_)=i}- (2-3) 

If we compensate Nij(t) by J A,j(s) ds, then the resulting process 

M„- (t) := % (t) - / \ tJ (s) ds (2.4) 

Jo 

is a purely discontinuous, sq uare-integrable martingale which is null at the origin (for example, see 
iRogers and Williams! 200Ct Lemma IV. 21. 12]). Note that the set of martingales {My;i,j E I,i =/= 



j} are mutually orthogonal. 

3 The control problem 

Suppose for some P G N we are given a set Li £ R and a control process u(t) — u(ui,t) : 
£2 x [0, T] — > U. We assume that the control u(t) is {J^j-adapted and cadlag. Consider the 
state variable X(t) — (Xi(t), . . . , Xpj(t)) T whose nth component satisfies the stochastic differential 
equation 

N 

dX n (t) = b n (t,X(t),u(t),a(t-))dt + J2 <Tn m (t,X(t),u(t),a(t^))dW m (t), (3.1) 

771—1 

where b n : [0 , T] x R N x R p x I -s- R and cr nm : [0,T]xM Jv xK p x/^lare given continuous functions 
for n, m = 1, . . . ,N. Using A T to denote the transpose of a matrix A, set b(t) := (&i(i), ■ ■ ■ , &Ar(t)) T 
and o-(t) := (cr„m(i))^ m= i. 

We consider a performance criterion defined for each x £ M. N as 



f-T 



J {u \x)~ jM(x,i ):=E[ / f(t,X(t),u(t),a(t))dt + h(X(T),a(T)) 



X(0) = x,a(0) =i 
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where for each i e I we have that /(•, •, •, i) : [0, T] x M. N x U — > R is continuous and fi(-, i) : M w — > R 
is C^R) and concave. 

We say that the control process u is admissible and write u € A if, for each x G M. N , p. II) has 
a unique, strong solution X(t) — X^(t), t S [0, T] satisfying both X(0) = x, a.s., and 

E( / f(t,X(t),u(t),a(t))dt + h(X(T),a{T))) < oo. 



V ° / 

The stochastic control problem is to find an optimal control u* £ A such that 

J("*)(x) = sup J {u \x). (3.2) 

Define the Hamiltonian "H : [0, T] x l w x U x / x R N x R Arxjv -> R by 

H(t,x,u,i,p,q) :— f(t,x,u,i) + b (t, x, u, i)p + tr(er (t,x,u,i)q), (3-3) 

where tr(A) denotes the trace of the matrix A. We assume that the Hamiltonian "H is differentiable 
with respect to x. 

The adjoint equation corresponding to u and X^ u ' in the unknown, adapted processes pit) £ R , 
g(f) S M JVxAr and rj(t) = (t7 (1) (£), . . . ,7 ? ( Ar )(i)) T , where t/") G R r,xl? for n = l,...,iV, is the 
backward stochastic differential equation 



(3.4) 



dpit) = -V x ft(f, AT(t), u(t), a(t),p(t), ?(*)) df + g T (f) dW(t) + 77(f) • dM(t) 
p(T) =V :c /i(X(T),a(T)), a.s. 

where V x W(t,X(t),u(t),a(t),p(t),g(t)) denotes V a ,W(* ) x,w(t),a(t),p(*),<?(t))| a;= x(t), 
V x h(X(T), a(T)) denotes V a; /i(a;,a(T))| : ,, = x(T) and, for notational convenience, we dchnc 

7,(4). dAf(f) := (XJf/^WdMo-W,... ,X)^(«)dM 4i (t) 

for all t € [0, T). Note that we use throughout this paper X^Vi as shorthand for 5Z i=1 $3j=i,- 

Remark 3.1. Notice that there are jumps in the adjoint equation Q3.4[) even though there are no 
jumps in the equation (13.1[) which governs the state variable Xty). This is a consequence of the 
coefficients b(t) and <j(t) being functions of the Markov chain a{t). Moreover, the unknown process 
77(f) in the adjoint equations ()3.4j) does not appear in the Hamiltonian (|3.3j) . 

4 Sufficient stochastic maximum principle 

Here we state and prove the sufficient stochastic maximum principle. In Section H3 we apply it to 
a quadratic loss minimization problem. 

Theorem 4.1 (Sufficient stochastic maximum principle). Let u E A with corresponding solution 
X = X^ u > and suppose that there exists a solution (p(i), q(t), 77(f)) of the corresponding adjoint 
equation \3.J$ satisfying 

T . -r 2 



E 
/o 



<7(f,X(f))-a(f,XM(f))) /5(f) 



df < 00, (4.1) 
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and 



q T (t)(x(t)~X^(t) 



dt < oo, 



d(Mij}(t) < oo, 



(4.2) 
(4.3) 



n=l jj&i 

for all admissible controls u £ A. Further suppose that 

1. n(t,X(t) 1 u{t),a(t) 7 p{t),q(t))^sup veU H(t 1 X(t),v,a(t),p(t),q(t)), Vt £ [0,2*], 

2. h(x,i) is a concave function of x for each i € I, and 

3. for each fixed pair {t,i) G [0,T] x /, H(x) := max„ £ y H(t,x,v,i,p(t), q(t)) exists and is a 
concave function of x. 

Then u is an optimal control. 

Proof. Fix u £ A with corresponding solution X = X^ u \ For notational ease, denote the quadruple 
(t,X(t-),u(t^),a(t^)) by (t,X(t_)) and similarly denote the quadruple (t,X(t-), u(t-),a(t-)) by 
(t,X(t-)). Then 

J{u) - J{u) = E I J (/(£, X{t)) - f(t, X(t))) dt + h(X(T),a(T)) - h(X(T),a(T)) j . 

We use the concavity of h(-, i) for each i £ I and (J3.4I) to obtain the inequalities 

E (h(X(T),a(T)) - h(X(T),a(T))) > E ((x(T) - X(T)) T V x h (x(T),a(T)) 

>e((x(t)-x(t)Yp(t) 

This gives 

J(u)-J(u)>EJ (/&*(*_)) -/(*,*(*_))) dt + E((x(T)-X(T)yp(T)Y (4.4) 

To expand the first term on the right-hand side of (|4.4p , we use the definition of Ti in (|3.3p to obtain 



E 



= E 



(f(t,X{t))-f(t,x(t)) 



dt 



E 



ft(i, X(t), fi(t), a(t)J(t),q(t)) - W(t, X(t),u(t), a{t),p(t)A{t))) dt 
(bit, X{t)) - b(t, X(t)j) T p(t) + tr (a(t, X(t)) - a(t, X(t))) T q(t) 



(4.5) 



dt 



To expand the second term on the right-hand side of (J4.4I) we begin by applying integration-by-parts 
to get 



(l(T)-X(T)) p(T)=/ (x(i)-X(i)) dp(t)+/ p T (t)d(!(i)-X(i)) + 



X-X,p 



(T). 
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Substitute for X, X and p from (|3.1[) and p.4[) to find 
(X{T)-X{T)Y p{T) 
= I (X(t)-X(t)) (-V^(i,l(i),ii(t),a(t),p(t),g(t))dt + q T (t)d^(t)+rKt). dAf(t)) 

+ / p T (*) ((&(*,*(*)) "&(*>*(*))) di+(a(t,X(i))-a(i,X(t))) T d^(i) X 

+ / tx($ T (t)(<K*,*(t)) -*(*,*(*)))) di. 



Due to the integrability conditions (|4.1j) - (|4.3j) . the Brownian motion and Markov chain martingale 
integrals in the latter equation are square-integrable martingales which are null at the origin. Thus 
taking expectations we obtain 



E 



(A-(T)-A-(T)) p(T) 
= E^ f- (X(t) - X(t)Y \7 x H(t,X(t),u(t),a(t),p(t),q(t))) d* 
+ E / (p T (t) (b(t,X(t))-b(t,X(t))) +tr (q T (t) (a(t,X(t))-a(t,X(t)) 



Substitute the last equation and (|4.5[) into the inequality (14.41) to find after cancellation that 

J(u) - J(u) > E f [H(t, X{t),u(t), a(t),p(t), q(t)) - H(t, X(t), u(t),a(t),p(t), q(t)) 

Jo V (4.6) 



- (X(t) - X(t)) T V x H(t, X(t), u(t), a(t),p(t),q(t))) dt 



We can show that the integrand on the right-hand side of (|4.6[) is non-negative a.s. for each t <G [0, T] 
by fixing the sta te of the Markov chai n and then using the assumed concavity of H(x) to apply 
the argument of iFramstad et al.l 20041 pages 83-84]. This gives J(u) — J(u) > and hence u is 



optimal. □ 

5 Connection to Dynamic Programming 

In a jump-diffusion setting, the co nnection between the st ochastic maximum principle and dynamic 



programming principle is shown in IFramstad et al.l 20041 Section 3]. We show a similar connection 



in Theorem l5.1[ between the value function V(t, x, i) of the control problem and the adjoint processes 
p(t), q(t) and rj(t). The main difference is that, in the regime-switching diffusion model, the adjoint 
process ijij (t) represents the jumps of the x-gradient of the value function due to the Markov chain 
switching from state i to state j. In the non-regime-switching jump-diffusion model, this adjoint 
process represents the jumps of the x-gradient of the value function due to the jumps in the state 
process X(t). 
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To put the problem in a Markovian framework so that we can apply dynamic programming, 
define 

J u (s, x,i):=F,( f (i, X(t),u(t), a(t)) dt + h(X(T),a(T)) X(s) = x, a(s) = % j , Vttei, 

and put 

V(s, x, i) := sup J u (s, x, i), (5.1) 

ueA 

for all {s,X,i) € [0,T] x R w x 7. 

Theorem 5.1. Assume that V(-, -,i) G C 1,3 ([0,T] x R ) /or eac/i i G 7 and that there exists an 
optimal Markov control u*(t,x,i) for H5.1)) , with corresponding solution X* = X^ u >. Define 

dV 
Pn(t)~g^(t,X*(t),a(t)), (5.2) 

N d 2 V 

q nm (t) :=^cn ro (t,X*(t), U *(i),a(i)) -^-^-(t 7 X*(t),a(t)), (5.3) 

i n) W := ^(t,^(i),j) - K.(t,X*(t),i). (5.4) 

Then p(t\ q(t) and rj(t) solve the adjoint equation Jff.^| ). 
Remark 5.2. To prove the above the orem, we require Ito's formula, which is given next. Ito's 



formula can be found in iProtterl [2005L Theorem 18, page 278] 



Theorem 5.3 (Ito's formula). Suppose we are given an N -dimensional process X = (X\, . . . , Xjsr) 
satisfying for each n= 1,...,N 



A 



dX n (t) = b n (t,X{t),a{t-))dt+J2 Vnm(t,X(t),a(t-))dW m (t) 



m—l 



X n (0) = Xq ' , a.s., 



r(") C TB n^rl &. m „t; nr ,e V 1 . . i\ CL C^^IWi T] N/ TCP N 



for some x ( n> G R, and functions V( v ,i) G C^QO.T] x M w ) /or eac/i i = !,...,£>. The 



V(t,X(t),a(t)) = V(Q,X(0),a(0))+ / rV(s,X(s),a(s_))ds 
r< dV 



>o 

N nt „, N 



i dV 
+ E / ^-(s,X(s),a(s-))y / a nm (s,X(s),a(s-))dW m (s) 



n— 1 u m—l 



** Jo 



+ ]T / (V(s,X(s)J) -V(s,X(s),i)) dMijit), 
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for 



dV , 1X A dV 



u v \ — ^ U V 

TV(t, x, i) := — (t, X, i) + 2_ a — (*> x , i)bn(t, X, i) 



dt ^-J dx n 

n—l 



1 x— -\ X^ d V x— ^ 

y^2 5Z a — a — (t,x,i)22o-„i(t,x,i)o- m i(t,x,'< 



Z OXjiOXyyi 

n—l m— 1 1=1 

D 
+ ^9x3 (V(jt,X,j) -V(t,X,i)) , 
3=1 

for all (t, x, i) € [0, T] x R N x I. 

Proof of Theorem I5.it From general dynamic programming theory, the Hamilton- Jacobi-Bellman 
equation holds: 

dV 

— (i, x, i) + sup {f(t, x, u, i) + A u V(t, x, i)} = 0, 
ot ueii 

where A u is the infinitesimal generator and the supremum is attained by u*(t, x, i). Dchnc 

dV 

Fit, x, u, i) := -r-(t, x, i) + f(t, x, u, i) + A u V(t, x, i). 
at 

Using Ito's formula ( Theorem 15. 3[) to expand A u V(t, x, i), we find 

dV v^> dV 

F(t, x, u, i) = f(t, x, u, i) + -rr-{t, x,i) + 2_^ - — (t, x, i)b n (t, x, i) 



Z ox n ox m 

n—l m— 1 ( = 1 

+ X] fffc '(V(t, a:,j) - V(t,x,i)) . 

3=1 

Differentiate F(t,x,u~*(t,x,i),i) with respect to x^ and evaluate at x = -^*(£) and i = a(t). For 
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notational ease denote the quadruple (t,X*(t),u*(t,X*(t),a(t)),a(t)) by (t,a(t)). We get 
N BV r)h 



9a;„ 9x fc 

N N „q Tr / W 



<9 3 U 



1 <9 a U / \ 



3=1 

Next define 



i w w a 2 ^ a / w \ 

2 E E a^^w')fe (5>* -J (i ' a(i)) 

n— 1 m— 1 " " \/ — 1 / 



<9U 
F fc (t :=— t,rt,at , forfc = l,...,J\T. 



Using Ito's formula ( Theorem 15. 3|) to obtain the dynamics of Yk(t), we find 



( ^2T/ Q2T/ 



Cfe " ax n ux k 

n—1 

A A 9 3 U 7 x 

2 ^-J ^ dx n dx m dx k , 

D 



1 OK I \ 

2 E E ^ft^^(*.^(*).«(*)) E— .»« ) (*.«(*)) 

n=lm=l " '" \/=l / 

f>(*)J (^(t,X*(i),j)-gl(t ) X*(i),a(i))) }di 



3=1 

JV q2 rr \ 



— d V — 

E fl 5 (*,**(*), a(*)) E ov. TO (*,a(t))dW OT (t) 



n=l 



dx n dxk 

f* X* t\ i\ 






(5.5) 
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Substituting for ® t Q x from (|5.5p . we get 

^ n— 1 n 

1 w w 9 2 V 9 / w \ 1 

+2E E to^:^^)^*))^ 5>H (*.«(*))}* 

n=l rn=l \J=1 / ' 



W c,2t, ^ 



E TT^T- (t,X*(t),<*(t)) E <fnm(t,o(t))dW m (t) 



Note that 



9cc n i9a;fe 

/ay. 

V 9x k ' '' dx k 



E (^(t,X*(t),j) - ^(t,X*(t),i)) dM«(t). 



'VVJlifv \_iJ^J^ pay ^ 

2 2-; Z. ^^ ^ I L,Wml\ 2 2^ 2_. ftp ft^ Z-, V /A,-,, <>,-,. 

n— 1 m— 1 \t= 1 / n— 1 m— 1 / — 1 

TV iV / iV 



da n i da m i 

&ml + <Jnl 



m=l t= 1 \n— 1 / 



Next, from f|3.3p we find that 



(5.6) 



(5.7) 



^(a(t),n(i) ) a(i) ) p(t) )( ;(t))=^(i ) a(i)) + ^^(i,a(t)K(t) 

n— 1 
n— 1 m— 1 

Substituting (J5.2I) - (|5.4I) . (I5.7[) and the last equation into (|5.6p gives 

dn(*) = -g^(t,X(t),u(t),a(t),p(t),q(t))dt+ E fa(t)dW m (t) +E^i ) (*)dM ij -(t), 

m— 1 j'^i 

and as !&(£) = Pfc(i) for each fc = l,...,iV, we have shown that p(i), q(i) and rj(t) given by 
(jIT2"j) - (p)l| solve the adjoint equation (|5H) . □ 

6 Application: quadratic loss minimization problem 

We demonstrate the use of the maximum principle by solving a quadratic loss minimization problem. 
Consider a regime-switching financial market that is built upon one traded asset, which we call the 
risky asset, and a risk-free asset. The risk-free asset's price process So — {So(t),t £ [0, T]} is given 
by 

4^M= r (i,a(i_))di, Vte[0,n 5 (0) = 1, (6.1) 
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where the risk-free rate of return r{t, i) is a bounded, deterministic function on [0, T] for i = 1, . . . , D. 
The price process Si = {S\(t),t G [0, T]} of the risky asset is given by 

^p^- = b(t,a(t-))dt + a(t,a(t-))dW(t), Vte[0,T], (6.2) 

with the initial value S'i(O) being a fixed, strictly positive constant in K. We assume that the mean 
rate of return b(t, i) and the volatility process a(t, i) are bounded, non-zero, deterministic functions 
on [0, T] for i = 1, . . . , D. Here, W is a 1-dimensional standard Brownian motion and b and a are 
scalar processes. 

A portfolio process ir{t) is a {J-t}-previsible scalar process which gives the amount invested in 
the risky asset at time t. Denote by 7To(i) the amount invested in the risk- free asset at time t. The 
corresponding wealth process X™{t) is then given by 

X 7r (t) = 7T (t)+7r(t). 

We assume that at time 0, X n (0) — xq, a.s. Define the market price of diffusion risk 0(t,i) :— 
cr~ l (t,i)(b(t,i) — r(t,i)). Under the self-financing condition, the dynamics of the wealth process 
satisfy 

dX n (t) = (r(t)X*(t) + w(t)a(t)9(t)) dt + n(t)a(t) dW(t), X ff (0) = x . (6.3) 

We say that 7r(£) is an admissible portfolio process and write n S A, if it is a {J^j-previsible, 
square-integrable, scalar process. 

We consider the problem of finding an admissible portfolio process n S A such that 

E(X ff (T)-d) 2 = inf E(X*(T)-d) 2 , 

for some fixed constant deR. 

To solve this, we use the sufficient maximum principle of Theorem 14.11 Define the real- valued 
function h(x) := —{x — d) 2 and consider the equivalent problem of maximizing 

E (h(X*(T))) =E(- (X n (T) - df) . (6.4) 

over all 7r G A. Set the control process u(t) :— n(t) and X(t) := X n (t). For this example, the 
Hamiltonian Q3.3P becomes 

%{t,x,u, i,p,q) ■= (r(t, i)x + ua(t, i)9(t,i))p + ua(t,i)q, (6-5) 

and the adjoint equations (J3.4I) are for all t e [0, T), 

dp(t) =-r(t)p(t)dt + q(t)dW(t)+J2 j ^r Hj (t)dM ij (t), 
p(T) =-2X(T) + 2d, a.s. { ' 

We seek the solution (p(t),q(t),r](i)) to (|6.6j) . Since h(x) is quadratic in x and the adjoint process 
p is the first derivative of the function h, a natural assumption is that p is linear in X. This means 
that p is of the form 

p(t) = 0(t, a(t))X(t) + ip(t, a(t)), (6.7) 
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where (f>(-,i) and ip(-,i) are deterministic, differentiable functions for each i = 1,...,D, which are 
to be found. From (|6.6p . <f> and tp have terminal boundary conditions 



4>(T,i) = -2 and ip(T,i) = 2d, Vie/. (6.8) 

The next step is to expand the right-hand side of (|6.7p and then compare it with (|6.6p . To do this, 
we begin by noting from Ito's formula (Theorem I5.3|) that for a function f(t,a(t)) we have 

df(t, a(t)) = f t (t, a(t-)) dt + J2 9a (/(*, j) - f(t, »)) l[a(t_) - t] di 

3 * 1 (6.9) 

+ ^(/(t,j)-/(i,i))dMy(t). 

Using (J6.9I) to expand the functions and ip, and (|6.3p to expand X (with 7r(i) := u(i) and 
X n {f) := X(i)), we apply integration-by-parts to (|6.7p to get 

D 

dp(i) = $jlL[a(t_)=*] 

X(t_) U(t,t)r(*,») + ^(t,t) + ^fti(0(t,i)-^(t,t)) 

+ ^(t, *)«(t)o-(*, »)»(«, o + Vt(*, o + J2 & w*> j) - ^c> *)) f dt 

+ <j)(t)u{t)cr{t) dW(t) 

+ y, (*(*-) m,j) - ct>(t,i)) + m,j) - 4,{t,i))\&M l3 {t) 

Comparing coefficients with (|6.6p . we obtain three equations 

= ^l[a(t_) = i]\x(t-) ct>(t,i)r(t,i) + <t> t (t,i) + J2to (4>(t,j)-^(t,i)) 

+ 0(<,t)«(«Mt,*)fl(t,t) + ^t(<,t) + 5^5ij W*.j)-^(*.*)) [. 

3=1 J 



(6.10) 



9 (t) = c)>(t)a(t)u(t), (6.11) 

»&•(*) = X(*_) (4>(t,j) - 0(i,i)) + (^(t,j) - ff>(t,i)) . (6.12) 

Let u G Abe a candidate for the optimal control with corresponding state process X and adjoint 
solution (p,q,fj). Then for the Hamiltonian (I6.5p . for all »eR, 

H(i, X(£), u, a(t),p(t), q(t)) = (r{t)X(t) + ua(t)6(tj) p(t) + ua(t)q{t). 
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As this is a linear lunction of u, we guess that the coefficient of u vanishes at optimality, which 
results in the equality 

q{t) = -0(t)p(t)- (6-13) 

Substituting into (|6.1ip for q(t) from (|6.13[) and using (|6.7p to replace j3(£), we get 



i(*) = -^(tWit) (X(t) + (j}-\t)^{t) 



(6.14) 

Therefore, to find the optimal control it remains to find <f> and ip. To do this, we set X(t) := X(t), 
u(t) :— u(t) and p(t) :— p(t) in (|6.10[) and then substitute for p{t) from (|6.7[) and for u(t) from 
(J6.14I) . This results in a linear equation in X(t). Assuming that the coefficient of X(t) equals zero, 
we obtain two equations 



D 

(f>(t, i) (2r(t, t) - \9(t, i)\ 2 ) + Mt, i)+J2 9H W*>i) ~ </>& *)) = °= 
il>(jt,i) (r(t,i) - |0(i,i)| 2 ) + V*(*.*) +53 S« (Tp(t,j)-ip(t,i)) = 0, 

3=1 

with terminal boundary conditions given by (|6.8[) . Consider the processes 



and 



4>(t,a(t)):=-2E [expi / (2r(s) - \9(s)\ 2 ) ds 



%i>(t,a(t)):=2dE[exp\ / (r(s) - \0{s)\ 2 ) ds 



a(t) 



a(t) 



(6.15) 
(6.16) 

(6.17) 
(6.18) 



We aim to show that <fi = <fi an d ip = ip. It is helpful to define at this point the following martingales: 

i-T 



and 



R(t) := E ( exp j / 
S(i) := E (exp j j 



{2r{s)-\9{s)\ 2 )ds 



(r{s) -\9(s)\ 2 ) ds 



J? 



J?], 



(6.19) 



(6.20) 



where J 7 " := a{a(r),T S [0, £]} V A/"(P) is the filtration generated by the Markov chain. From the 
{J^j-martingale representation theorem, there exists {J-" t Q }-previsible, square- integrable processes 
u R (t),u s (t) such that 

R(t) = R(0) + W "§(t) dA%(r) and S(t) = 5(0) + W i/*(r) dM y (r). 

. -j.- JO j_u JO 



&i 



j¥* ' 



By the positivity of R(t) and S(t), we can define the processes &$(£) := v^{t)R l {t—) and vfAt) 
^•(t)5 _1 (*-) so that 

fl(«) = i?(0) + V f R(t-)v%(t) dM^(r) and S{t) = 5(0) + V f S(t-)v%(t) dM y (r). 



(6.21) 
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From (|6.17[) and the definition of R in (|6.19p . we have the relationship 

i?(t) = -^(*,a(t))exp|y (2r(s)-\e( s )\ 2 )d S \ : We[0,T]. (6.22) 

Using the Ito formula expansion of <p(t,a(t)) (see (|6.9p ). we apply integration-by-parts to expand 
the right-hand side of the above equation and comparing it with the martingale representation of 
R(t) given by (J6.21I) . we find that </> satisfies (|6. 15|) with <p :— <p. We conclude that <f> = <j>. 
Similarly, from (|6.18|) and the definition of S in (|6.20|) , we have 

S(t) = ^(t 7 a(t)) CX pH^(r( S )-\8( S )\ 2 )d s y We[0,T]. (6.23) 

Using the Ito formula expansion of ip{t,a(t)) (see (|6.9[) ) . we apply integration-by-parts to expand 
the right-hand side of the above equation and comparing it with S(t) given by (|6.2ip . we find that 
tp satisfies (|6.16|) with ip := ip. We conclude that ip = ip. Thus from (|6.7p . (|6.1ip and (|6.12p . we 
can write down the solutions 

p(t) = <p{t)X{t)+ip{t), q{t) = <t>{t)<j{t)u{t), fj i:j (t) = x(t_) (cf>(t,j) - 0(t,*))+(^(*.i) - ^(*.*)) • 

to the adjoint equation (|6.6p . Substitute into (J6.14I) for <fi = <p from (|6.22p and for ip = ip from 
(J6.23I) and use the Markov property of a to obtain the control process 



u(t) 



( E(^exp|/ ( T (r( S )-|0(.s)| 2 )d.s||a(^ 



X(t) -d 



\ 



\ 



E(expjf t T (2r( S )-|0( S )P)d S j 



a(t) 



- l (t)9(t). (6.24) 



With this choice of control process and the boundedness conditions on the market parameters r, b 
and a, the conditions of Theorem 14 . 1 1 are satisfied and hence u(t) is the optimal control process. 

Remark 6.1. The above result can be used to obtain the solution to the classical problem of mean- 
variance portfolio optimization. Suppose we wish to find an admissible portfolio process which 
minimizes vax(X(T)) = E(X(T) - E(X(T))) 2 subject to E(X(T)) = a, for some a £ R. Applying 
a Lagrange multiplier technique, we note that for all A 6 K, 

E ((X(T) - a) 2 + 2X(X(T) - a)) = E (X(T) - a + \f - A 2 . 

Fix AeR and minimize E {X(T) — a + A) . The portfolio process which minimizes this is u(t) := 
u(t;X), which is given by (|6.24p with d := a — A. Then we maximize the quadratic function 
E (X(T) — a + A) — A 2 over all A £ K to find the optimal A* £ R and hence we obtain the optimal 
portfolio process u(t; A*) which solves the mean- variance problem. 



Remark 6.2. The optimal control process for the mean- variance problem was also found in lZhou and Yir 
|2003l | using a stochastic LQ control technique and completion-of-squares. 
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